University of Sydney


The limitations of observational data


Alternatives to observational data?


Experiments

Experiments

Why experiments?

  • Experiments help establish causation.
  • Experimental research has real policy implications.

What are some problems with experiments?

  • Experiments can be difficult to run and expensive.
  • Many are unethical.

Becoming increasingly popular

Becoming increasingly popular

Types of experiments

  • Labratory.

Lab experiments

What is a lab experiment?

  • Expose subjects to treatment in controlled (artificial) setting and observe reactions.
  • Complete control of environment (very high internal validity).
  • Does the lab replicate a 'real world' environment? (lower external validity).

What are some examples?

  • Stanford Prison Experiment, Philip Zimbardo.
  • Effects of 'irrelevant stimuli' on responses to policy prompts about immigration.

Effects of 'irrelevant stimuli' on responses to policy prompts

What did this experiment involve?

  • 224 subjects read and responded to policy statements. Eg, asked to respond to the statement: 'The number of illegal immigrants coming to the United States will drastically increase in six years.' Responses rated as positive or negative.
  • Subjects’ responses to policy prompts tended to reflect preexisting views.
  • However, difference dwarfed by difference between subjects with similar views exposed to different 'irrelevant stimuli'.
  • Exposure to positive stimuli = 42% fewer negative and 160% more positive thoughts about illegal immigration than negative.

Effects of 'irrelevant stimuli' on responses to policy prompts

What were these 'irrelevant stimuli'?

  • This is what makes this an experiment.
  • Random treatment: Before seeing each policy statement, each subject was subliminally exposed (for 39 milliseconds, well below the threshold of conscious awareness) to: a smiling, frowning or neutral cartoon face.
  • Quick exposure to images of cartoon smiley faces had a larger effect than prior immigration attitudes on the valence of thoughts in response to illegal immigration policy prompts.

Lab experiments

What is a lab experiment?

  • Expose subjects to treatment in controlled setting and observe reactions.
  • Complete control of environment (very high internal validity).
  • Does the lab replicate a 'real world' environment? (lower external validity).

What are some examples?

  • Stanford Prison Experiment, Philip Zimbardo.
  • Effects of 'irrelevant stimuli' on responses to policy prompts about immigration.
    • Problems with these kinds of research design?

Types of experiments

  • Labratory.
  • Field.

Field Experiments

What is a field experiment?

  • Research in a realistic setting. Treatment intervenes and manipulates independent variable(s) and controls situatation as well as possible.
  • High external validity.
  • Less control of environment, leading to lower internal validity.

What are some examples?

  • The impact of school lunches on performance.
  • AB testing for websites.
  • Campaign advertising.

Testing the efficacy of campaign advertising


  • In 2005 Republican consultant Dave Carney hired political scientist Donald P. Green to run a series of random trials for Texas Governor Rick Perrys (primary) re-election campaign.
  • Spent $2 million in positive, randomly assigning television and radio ads, to 18 different markets over three weeks.
  • Found short-term effect of TV ads: increased Perrys numbers by 5% for ~ a week.

Types of experiments

  • Labratory
  • Field
  • Survey

Survey experiments

What is a survey experiment?

  • Can be a subset of a field and lab experiment.
  • Apply a treatment on a random subset of the sample then ask a series of questions. The random treatment is what makes these experiments.
  • Treatment can be another question.

Survey experiments

For instance. Want to understand the causal impact of economic insecurity on support for populism.

Survey experiments

Are you extremely worried, very worried, moderately worried, a little worried or not at all worried about:

  1. Losing your job in the next 12 months.
  2. Having enough money to retire on.
  3. Paying your rent or mortgage.

Survey experiments

We then ask a series of questions designed to tap into populist or authoritarian attitudes, such as:

Do you strongly agree, agree, disagree or strongly disagree that:

  1. Politicians need to follow the will of the people.
  2. Most politicians are competent people.
  3. The elite have different political views to ordinary people.

<1> Strongly agree <2> Agree <3> Disagree <4> Strongly disagree

Survey experiments

What is a survey experiment?

  • Can be a subset of a field and lab experiment.
  • Apply a treatment on a random subset of the sample then ask a series of questions. The random treatment is what makes these experiments.
  • Treatment can be another question.
  • Also can be statements or media, or even ordering of questions.
  • List experiments arguably a subset of this.

Problems with survey experiments?

Types of experiments

  • Labratory.
  • Field.
  • Survey.
  • Natural.

Natural experiments

What is a natural experiment?

  • Using a treatment that is 'random' but not controlled by the researcher to understand causality.
  • Lotteries, natural disasters, Media markets.

Natural experiments

Natural experiments

What is a natural experiment?

  • Using a treatment that is 'random' but not controlled by the researcher to understand causality.
  • Lotteries, natural disasters, Media markets.

Problems with natural experiments?

  • We don't control the application of the treatment.
  • The treatment may not be truly exogenous to the outcome we are studying.

Using spatial data to understand the world


What is spatial data?


Why and how do we use spatial data?

Why do we use spatial data?

  • We want to understand human behavior, and processes involving humans and their actions, predict these behaviours and processes, and find solution to problems that face society.
  • Much of human behaviour can be understood geographically.
  • Tobler’s First Law of Geography; spatial dependence. Basis of much spatial analysis.
  • Spatially defined groups (towns, cities, states, countries) often have a great deal of social, historical and economic meaning. Not merely defined by their distance from one another.

Why do we use spatial data?

  • Spatial analysis examines data in cross-section, as opposed to longitudinal analysis (although the two can be combined).
  • There is often a great deal of data on spatially defined groups (towns, cities, states, countries). Ie from census.
  • Spatial analysis can play important role in inductive and deductive approaches to science. Inductive: display of data in spatial context may reveal patterns and anomalies and suggest processes explaining them (ie John Snow, 1854).

John Snow, mapping the Cholera outbreak of 1854

Why do we use spatial data?

  • Spatial analysis examines data in cross-section, as opposed to longitudinal analysis (although the two can be combined).
  • There is often a great deal of data on spatially defined groups (towns, cities, states, countries). Ie from census.
  • Spatial analysis can play important role in inductive and deductive approaches to science. Inductive: display of data in spatial context may reveal patterns and anomalies and suggest processes explaining them (ie John Snow, 1854).
  • Deductive approach to test theories and principles. Recognition of confounding factors.

Limitations


  • Boundaries must make sense, conceptually, methodologically.
  • Must have access to data that answers our question.
  • Ecological fallacy.

How do we use spatial data?

  • Tables.

Tables

Vote share of Republican Donald Trump at the 2016 US presidential election, and population density, by state
State Trump vote share (%) Pop density (per sKm)
Alabama 62 37
Alaska 51 0
Arizona 49 23
Arkansas 61 22
California 32 97
Colorado 43 20
Connecticut 41 286
Delaware 42 187
District of Columbia 4 4251
Florida 49 145
Georgia 51 68
Hawaii 30 86
Idaho 59 7
Illinois 39 89
Indiana 57 71
Iowa 51 21
Kansas 57 14
Kentucky 63 43
Louisiana 58 41
Maine 45 16
Maryland 34 238
Massachusetts 33 336
Michigan 48 67
Minnesota 45 26
Mississippi 58 24
Missouri 57 34
Montana 56 2
Nebraska 54 9
Nevada 46 10
New Hampshire 47 57
New Jersey 41 470
New Mexico 40 6
New York 37 162
North Carolina 50 79
North Dakota 63 4
Ohio 52 109
Oklahoma 65 22
Oregon 39 16
Pennsylvania 48 110
Rhode Island 39 394
South Carolina 55 62
South Dakota 62 4
Tennessee 61 61
Texas 52 40
Utah 46 14
Vermont 30 26
Virginia 44 81
Washington 37 41
West Virginia 69 29
Wisconsin 47 41
Wyoming 67 2

How do we use spatial data?

  • Tables.
  • Graphs.

Graphs

How do we use spatial data?

  • Tables.
  • Graphs.
  • Maps.

Maps

Types of maps

  • Standard.
  • Choropleth.
  • Distorted.

Standard map

Choropleth map

However, this kind of map does not always work

Distorted (choropleth) map

Types of maps

  • Standard.
  • Choropleth.
  • Distorted.
  • Dot.

Dot map

Types of maps

  • Standard.
  • Choropleth.
  • Distorted.
  • Dot.
  • Technical advances (GIS in particular, and interactive).

Operating in the R environment

Contact